Extract Transform Load (ETL) tools allow users to specify how to transform data from a variety of input formats to an output format that can be used in enterprise processes. Such tools allow users to define a set of transformation rules, either in graphical form or in a C-like language, resulting in a process that can be applied to input data to perform the transformations.
However, a drawback of existing ETL tools is that processes generated by them are restricted to a particular back-end. It is desirable to be able to translate ETL transformation flows to multiple back-ends.
For instance, in streaming applications (for example, on InforSphere Streams) one may need to ingest data from a relational database, combine it with streaming data (for instance, coming from sensors) and process/transform it in the Stream Processing Application Declarative Engine (SPADE) language. To do this, part of the transformation flow will have to be translated to structured query language (SQL), while part of the transformation should be translated to SPADE. This is not possible with existing tools. Furthermore, it is also desirable to automatically decide how to partition transformations across platforms (for example, what goes into SQL and what goes into SPADE) based on user-defined policies. However, this is also not possible with existing tools.